Neural Information Processing Systems http://nips.cc/

Existing MIPL algorithms often overlook the margins for attention scores andpredicted probabilities, leading tosuboptimal generalization performance.

Theoretical and empirical studies [21,77,9,55,49,5,12]postulate that such flatter regions generalize better than sharper minima, e.g., due to the flat minimizer's robustness against loss function shifts between trainandtestdata,asillustrated inFig.1.

For structured pruning, one promising method is to associate each neuron with a scaling factor, and regularize these scaling factors in training.

Our bound addresses the second question; it suggests that learning algorithms that bias towards models with small variation across the source threat model exhibit smaller drop in robustness to particularunforeseenattacks.

Wefindthatasthespectraldecay varies, the (asymptotic) test MSEisindeed approximately 2 inagreementwithpart (b) of Theorem3.1.